The present disclosure relates generally to computers. More particularly, the present disclosure relates to transactions in non-standard data segments in data block processing systems.
Many large computer systems can simultaneously run multiple operating systems (OSs). One issue with such arrangement is effectively allocating system resources to the multiple OSs. Some computers that support multiple OSs fix resource allocations for each OS, where the allocations do not change over time. More complex computer systems can dynamically allocate resources based on the needs of each OS.
Virtualizing memory functions has become a topic of great interest. Virtualizing memory can be done many ways. One way is by “paging” memory or swapping memory, where underutilized information stored in high performance memory is flushed out or moved to a remote location such as onto a disk. The term paging is often utilized because it is a flushing technique where the virtual address space is divided into fixed-sized blocks called pages and pages are flushed. A “page” is generally a contiguous block of memory of a certain size where a system manages units or blocks of memory the size of a page. Traditionally, pages in a system have uniform size or segment, for example, 4096 bytes. A page is generally the smallest segment or unit of translation available to an OS. Accordingly, a page is a definition for a standardized segment of data that can be moved and processed by the system.
As part of virtualizing memory, traditional OSs perform memory pinning or pinning of memory pages. Pinning, of pages or reserving of memory pages is typically a privileged capability. Pinning is typically not available to normal processes but only to hypervisors and kernel extensions of the OS. A kernel extension can be a part of an OS that coordinates sharing of resources, where a kernel can “take” resources from the OS.
It can be appreciated that to make multiple OSs on the same platform operate efficiently, proper memory management can be a concern. On a computing platform that runs multiple OSs and/or processes, a majority of OSs and/or processors may stay idle, only utilizing a small percentage of their resources, while a few OSs or processors run at or over capacity. An OS that is overcommitted or running over capacity can be slowed by the lack of resources and can even fail. It can be appreciated that memory allocation is an important feature and lack of memory resources can significantly affect a platform and an OS.
For example, when an OS runs out of real or high performance memory assets such as random access memory (RAM), the OS will often resort to a mass data storage asset such as a disk drive that, compared to RAM is slow and inefficient. If the memory required by the OS or application exceeds what is allocated in RAM (for example, due to unpredicted high peak demand), and the system cannot locate and utilize additional memory resources, the OS can run out of memory and the system can lock up or “crash.”
To avoid memory shortcomings, some computing platforms provide virtual memory to OSs. Generally, a virtual memory can be memory that is allocated but which may not have a “physical” basis at any time, but can be mimicked as physical memory in conceptual terms. Alternately described, virtual memory can be a subsystem which is simulated by the computing platform. Thus, virtual memory can describe an amount of memory that is larger than the actual memory. However, a properly operated virtual memory system can provide more cost effective storage than a memory system with fixed memory allocations.
Many virtual memory systems are controlled by a hypervisor or a virtual memory manager (VMM). A hypervisor is often referred to as a virtual machine monitor/controller, where a hypervisor can manage a system such that multiple OSs can share resources according to a set of rules. Typical virtual memory designs overcommit memory.
OSs that run in a virtualized memory environment may not be aware of, memory virtualization. Some traditional systems provide memory virtualization by using an “off-the-shelf” operating system and adding a cooperative memory overcommitment (CMO) kernel extension to the OS. This kernel extension can assist a hypervisor in managing the virtualized OS's memory resources. By limiting the amount of memory that can be pinned, the system is left with a large enough pageable pool to manage future allocations.
Global collaboration is becoming increasingly important in today's society. More and more applications are being designed to allow application data to be shared by multiple users across the globe or within the given network. Currently, methods exist that allow for this data sharing but with drawbacks. For instance, consider two users needing to modify a common file. This file can be shared on a network by both users, but with a price. Users currently have to setup a network file system (NFS) to host the file, then in order to access the data, information will have to be transmitted across the network. If security is an issue, a secure system must be provided. In order to share one file, users must now utilize multiple technologies in order to get access to the file.
Many existing distributed computing solutions implement a master-slave architecture where there exists a master node which sends workloads to the slave nodes. The master node can collect the results from each slave node as a given task is completed. In order for this communication to take place, the master node must serialize the data objects or pass the data in a primitive form to be parsed/deserialized by the slave nodes. This process is time consuming since it requires significant overhead to parse the data.